This application claims benefit of Japanese Application No. 2000-044903 filed in Japan on Feb. 22, 2000, and Japanese Application No. 2000-044904 filed in Japan on Feb. 22, 2000, the contents of which are incorporated by this reference.
1. Field of the Invention
The present invention relates to an image processing apparatus and, more specifically, to an image processing apparatus for adjusting a gradation range of an input image.
2. Related Art Statement
In an image processing apparatus for synthesizing a plurality of images picked up under different exposure conditions to generate one wide dynamic range image, a technique for adjusting a gradation range is used. For example, in Japanese Patent Application No. 11-338551 which has not been published yet, an image process device for dividing each of images into an appropriate exposure region and an inappropriate exposure region, performing gradation correction for each appropriate exposure region, and synthesizing the appropriate exposure regions of the images subjected to the gradation correction with each other to generate one wide dynamic range image is disclosed. In addition, as an example of an apparatus to which the image processing apparatus is applied, a super latitude digital camera being capable of picking up an image of an object in a wider dynamic range is described.
A gradation converting process in the image processing apparatus described above is performed on the basis of histogram flattening of an edge portion. This histogram flattening is a technique which is on the assumption that a main object as a large number of edges, the other portions such as a background has a small number of edges.
On the other hand, in a conventional digital camera, a color difference signal is also converted on the basis of a coefficient obtained when a luminance signal is subjected to gradation conversion. More specifically, it is assumed that a luminance signal Yorg is converted by a gradation conversion characteristic F such that Ytra=F (Yorg) is satisfied. In this case, conventionally, a conversion coefficient gain of a luminance signal is calculated by:
gain=Ytra/Yorg, and 
the conversion coefficient is directly used to convert color difference signals as follows:
Cbtra=gainxc2x7Cborg 
Crtra=gainxc2x7Crorg. 
In a digital camera, a bit count obtained when an image signal output as an analog signal from a CCD into a digital signal is often a bit count obtained when the image signal is processed and then recorded on, e.g., a recording medium.
In this case, even though a single input image is handled to perform conversion of an amount of information, especially, a reduction of an amount of information, the gradation range of the image may have to be adjusted.
The technique of gradation correction is popularly handled in various devices such as a printer device or a monitor device which is not limited to the digital camera and processes an image. The above technique can be applied to not only a case in which a plurality of images are handled but also a case in which a single input image is handled.
The technique of histogram flattening on the assumption that the main object as described above has a large number of edges can correspond to an object in a relatively wide range. However, the technique cannot completely correspond to the object. As an example which may correspond to the exception, a case in which a person is photographed in a relatively small size on a background having a plurality of shapes or contours is used. At this time, the background is decided as a main object to detect a large number of edges from the background portion, and a gradation range which is assigned to a person is reduced.
In a technique which performs gradation conversion to the color difference signal as described above by using a conversion coefficient equal to the conversion coefficient of a luminance signal, unnatural colors may be generated by a high-luminance portion. More specifically, color reproduction in a color space (e.g., Y, Cb, and Cr space) has a theoretical limit characteristic (see FIG. 11 showing an embodiment of the present invention). The theoretical limit characteristic of the color reproduction is a characteristic in which a color difference range in which colors can be reproduced increases as a luminance Y is increased, and a color difference range in which colors can be reproduced when the luminance exceeds the luminance Y. More specifically, a color reproduction range is narrow because the colors become blackish as a whole at a low luminance, colors in a wide range can be reproduced at an appropriate luminance, and a color reproduction range becomes narrow again because the colors becomes whitish as a whole at a high luminance.
When gradation conversion is performed like the case of luminance without consideration of the color reproduction range, the color reproduction range is close to or exceeds the limit of the color reproduction range, and colors after the gradation conversion may be whitish. In order to cope with the problem, a process of suppressing a saturation of a high-luminance portion is added in a conventional technique. However, it is not sufficient, and the realization of a technique of more improving color reproduction is desired.
It is an object of the present invention to provide an image processing apparatus being capable of adaptively adjusting a gradation range of a main object in accordance with a photographic scene.
It is another object of the present invention to provide an image processing apparatus being capable of adjusting a more optimum saturation in consideration of the theoretical limit characteristic of color reproduction.
In short, the present invention is an image processing apparatus, for adjusting a gradation range of an input image, including: photographing condition estimation means for estimating a photographing condition of the input image; and gradation correction means for performing gradation correction on the input image on the basis of the photographing condition to adjust the gradation range to a predetermined gradation range.
The present invention is an image processing apparatus, for adjusting a gradation range of an input image, including: luminance-color difference separation means for separating the input image into a luminance signal and a color difference signal; gradation correction means for performing gradation correction on the luminance signal to adjust the gradation range to a predetermined gradation range; color difference correction means for correcting the color difference signal on the basis of a luminance signal obtained before the gradation correction and output from the luminance-color difference separation means, a luminance signal obtained after the gradation correction, and a theoretical limit characteristic of color reproduction; and luminance-color difference synthesis means for synthesizing the luminance signal obtained after the gradation correction and a color difference signal obtained after the correction into an image signal.
Furthermore, the present invention is also directed to an image processing apparatus for adjusting a gradation range of an input image, including: photographing condition estimation means for estimating a photographing condition of the input image; luminance-color difference separation means for separating the input image into a luminance signal and a color difference signal; gradation correction means for performing gradation correction on the luminance signal on the basis of the photographing condition to adjust the gradation range to a predetermined gradation range; color difference correction means for performing correction on the color difference signal on the basis of a luminance signal obtained before the gradation correction and output from the luminance-color difference separation means, a luminance signal obtained after gradation correction and output from the gradation correction means, and a theoretical limit characteristic of color reproduction; and luminance-color difference synthesis means for synthesizing an image signal on the basis of the luminance signal obtained after the gradation correction and a color difference signal obtained after the correction.
The present invention is also directed to an image processing apparatus for processing an image group comprising a plurality of images obtained by image pickup to the same object under different exposure conditions to generate one wide dynamic range image, including: photographing condition estimation means for estimating a photographing condition; extraction means for extracting an appropriate exposure region on the basis of an image signal level of each image in the image group; gradation correction means for performing gradation correction on the appropriate exposure region on the basis of the photographing condition; and synthesis means for synthesizing appropriate exposure regions subjected to gradation correction by the gradation correction means to generate one wide dynamic range image.
In addition, the present invention is directed to an image processing apparatus for processing an image group comprising a plurality of images obtained by image pickup to the same object under different exposure conditions to generate one wide dynamic range image, including: luminance-color difference separation means for separating an image signal of each image in the image group into a luminance signal and a color difference signal; extraction means for extracting an appropriate exposure region on the basis of a signal level of the luminance signal; gradation correction means for performing gradation correction on a luminance signal of the appropriate exposure region; color difference correction means for performing correction on a color difference signal of the appropriate exposure region on the basis of a luminance signal obtained before the gradation correction and output from the luminance-color difference separation means, a luminance signal obtained after gradation correction and output from the gradation correction means, and a theoretical limit characteristic of color reproduction; luminance-color difference synthesis means for synthesizing an image signal on the basis of the luminance signal obtained after the gradation correction and a color difference signal obtained after the correction; and synthesis means for synthesizing the synthesized image signal of the appropriate exposure regions to generate one wide dynamic range image.
The present invention is still further directed to an image processing apparatus for processing an image group comprising a plurality of images obtained by image pickup to the same object under different exposure conditions to generate one wide dynamic range image, including: photographing condition estimation means for estimating a photographing condition; luminance-color difference separation means for separating an image signal of each image in the image group into a luminance signal and a color difference signal; extraction means for extracting an appropriate exposure region on the basis of a signal level of the luminance signal; gradation correction means for performing gradation correction on the luminance signal of the appropriate exposure region on the basis of the photographing condition; color difference correction means for performing correction on a color difference signal of the appropriate exposure region on the basis of a luminance signal obtained before the gradation correction and output from the luminance-color difference separation means, a luminance signal obtained after gradation correction and output from the gradation correction means, and a theoretical limit characteristic of color reproduction; luminance-color difference synthesis means for synthesizing an image signal on the basis of the luminance signal obtained after the gradation correction and a color difference signal obtained after the correction; and synthesis means for synthesizing the synthesized image signal of the appropriate exposure regions to generate one wide dynamic range image.
In addition, the present invention is directed to a recording medium on which a process program executed by a computer is recorded, wherein the process program adjusts a gradation range of an input image and includes the step of separating the input image into a luminance signal and a color difference signal; the step of performing gradation correction on the luminance signal to adjust the gradation range to a predetermined gradation range; the step of performing correction on the color difference signal on the basis of a luminance signal obtained before the gradation correction, a luminance signal obtained after the gradation correction, and a theoretical luminance characteristic of color reproduction; and the step of synthesizing an image signal on the basis of the luminance signal obtained after gradation correction and a color difference signal obtained after the correction.
The present invention is also directed to a recording medium on which a process program executed by a computer is recorded, wherein the process program processes an image group comprising a plurality of images obtained by image pickup performed to the same object under different exposure conditions to generate one wide dynamic range image and includes: the step of separating an image signal of each image in the image group into a luminance signal and a color difference signal; the step of extracting an appropriate exposure region on the basis of a signal level of the luminance signal; the step of performing gradation correction on a luminance signal of the appropriate exposure region; the step of performing correction on a color difference signal of the appropriate exposure region on the basis of a luminance signal obtained before the gradation correction, a luminance signal obtained after gradation correction, and a theoretical limit characteristic of color reproduction; the step of synthesizing an image signal on the basis of the luminance signal obtained after the gradation correction and a color difference signal obtained after the correction; and the step of synthesizing the synthesized image signal of the appropriate exposure regions to generate one wide dynamic range image.
These objects and advantages of the present invention will become further apparent from the following detailed explanation.